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ABSTRACT * 

The standard error of measurement (SEM) is a measure 
of^the* inconsistency in the scores of a part icular\group of 
test-takers. ' It i sr- largest for test-takers with scores ranging in the 
50 percent correct bracket; ^with nearly perfect sc'ores, it is . 
smaller. On tests used to make pass/fail decisions, the test-takers' 
scores tend to cluster in the range of 80-90 percent correct, with 
the passing score in the range of 60-70 correct. In this case, the 
*SEM, for the full group of test*-takers will be much smaller than the 
SEM for those with scores near the passing score. But, the * 
test-takers with scores near the passing score are the ones for Whom 
the reliability of the test is the most important, for them, 
measurement errors can make the difference between passing and 
failing. For this reason, the infportant SEM is not the SEM for the// 
full group of test-takers, rather, tne SEM at the passing score, 
which will often be substantially larger. A formula for^this, and its 
derivation, are provided. f CAuthar/CE) , * 
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ABSTRACT 

For tests used to make pass/fail decisions, the relevant standard error 
of measurement is the SEM at the passing score. If the test is highly 

stratified, this SEM should be estimated by a split-halves approach. A 

1 



formula and its derivation are provided. 
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Estimation of the Conditional 
Standard Error of Measurement 
for Stratified Tests 



The standard error of measurement (SEM) is ,a measure of the inconsistency 
in the scores of t a particular group of test-t-akers. It is Largest for test- 
takers with scchres* in the range of 50 percent cbrr^ct;. rt is much- smaller for 
test-t.akers with Kearly perfect scores. On many tests used to make pass/fail^^ 
decisions, the test-takers' scores tend to cluster in the range of 80 to 90 
.percent correct , while 'the passing, score tends to be in the lower tail of the 

\ . i r 

distribution, in the range of 60*to 70 percent correct. In this case, the SEM- 
for the full' group of test-takers will be much^smaller than the. SEM for those 
test-takers with scores near the passing score. But the test-takers with 
scores'near the passirfg score are the ones for whom the reliability of the test 
is most important. For them, error of measurement can make the difference 
between passing and (ailing. Therefore, when a test is \ised to make pass/fail 
decisions ,. 'the 'important SEM is not the SEfo for the full group of test-takers, 
bdt the SfeM at the passing score', which will often be substantially larger. 

, One simple solution to this problem is to estimate the, SEM at the, passing 
score by the formula for the standard deviation of a binomial distribution: 

SEM.P = V?<M - P)/M 

wtjere-P is the passing soore (number of correct answers) and.M is the maximum - 

% * < 

possible score, i.e.., the number of questions on the test (6ee Lord, '1957). 
This, solution' considers the SEM as the standard error of the sum of \ simple 
random sample of items. For many t^s.ts this assumption may 'be reasonably close 
to 'the truth/ But on ^ny t^sts the content is .highly stratified and the 
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test-takers' knowledge varies considerably fr6m one content category to another. 

' \ '<? 

In these cases, the binomial formula will tend to overest imate the SEM at the 
passing score*, , w \- 

Why does the binomial formula overestimate the cpnditional SEM for strati- 
fied tests? It assumes that a test-taker's probability of answering correctly 

is the same for all questions on the'test. But if the test content is highly 

j» . * . * * 

stratified, this assumption is likely to be qinte wrong f or* many testrtakers. 

Suppose • ^ 

For example, e uppogfc a test contains ten questions, t-w# from each of five 

content areas. And suppose a test taker knows the right answers to 90 per cent 
of the questions in the first content area, 30 percent in the second, 70 
percent in the third, 20 percent in the fourjth, and 90 percent in the fifth, it 
The SEM for this test-taker is actually given by 



0)(.10) + 2(.30)(.70) + 2(. 70)0.30) + 2( «0)(.80) + £(.90)(.10) 



.= 1.23. 



but the binomial approach yields the estimate 



VlO C.60\(.40) 



= 1.35 



C. 



because the test-taker's ^expected total score is six questions, or 60- percent 
correct. • 

The usual procedure for Estimating th£ ^overall SEM (i.e., for the full group 

of test-takers) on highly^stratif ied tests is to use .the split-halves method. 

The purpose of this paper is to propose an adaptation of the split-hal.ves 

method for .estimating the SEM at the passing score. This solution uses the 4 
* * • 

data, b$pm, only those test-takers with test scorefe at the passing scor,e. To 
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estimate the^ SEM .for these t«st-t aker Is by the split-halves method, first split 

the test inlo halves as similar in content and difficulty as possible. Then 

compute separate half-test scores for each test-taker whose total (full test) 

score is at fhe passing score. Let and X 2i represent the two half-test 

* scores for the i th test-taker, and let n represent the number of test-takers 
/ % * * ) 

with scores at the passing scdre. Then estimate the SEM at the passing score 



,by the formula: 



. v. 




(x u - x 21 ); 



The derivation of this formula is based on the assumption that, for 
any individual test-taker, the scores on the two half-tests are independent 
random variables with th% same me an^fid' variance. For any given test-taker, 
.the SEM^will be the square root of 

Var a u + X 2i ) 



\ 



= Var (X^) + Var (X 2i ) + 2 Gov (X^, X 2i ) 



= 2 Var (X 1± ) 



because c of .the assumptions of independence and equal variance of the r - hal f-tes t 

N O i M 



scores . 



To estimate Var (X,/) from the two half-test scores we can use the 



formula 



N 
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where N = 2 (for the two half-tests) and Y is the average of the r two half-test 

V 

scores. Thus our estimate of the variance of is 

4 * . [x u - Js(x u + x 2i )[ 2 + [x 21 - Js( Xli + x. 2i )} 2 . " * - . 

■[ Wx ii r x 2i>] 2 + [ j5(x 2i: x ii ) ] 2 . 

- • - 2 . 

n. 13 ^li " - X 2i^ '* * * * e << 

This estimate refers tQ an individual test-taker. It can be averaged over all 

</..' 
test-takers with total scores at' the passing £core , yielding the estimate 

• n 

2 ' 



^ (x ii } -^rl (x ii->> 



i=i - 

Since ttve variance of the tptal test score is twice the variance of each 
half-test score, we <have the estimate * 

(X u + X 21 > « 2 (X u ) .'^ 
n * 

: (x ii" x 2i )2 • 

i=i > 

The square root of-^his quantity will provide an^ estimate of tl>e SEM at the 
passing score. 1 , , f 
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